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1 - Introduction. 



Aim of this paper is to develop a new technique, based on the Baire category theorem, 
in order to estabhsh the closure of reachable sets and the existence of optimal trajectories 
for control systems, without the usual convexity assumptions. 

Most of our results will be formulated within the framework of differential inclusions. 
Let F : IR X 2^ be a continuous multifunction with compact convex vahies and 

denote by extF(t, x) the set of extreme points of F{t, x). We say that F has the bang-bang 
property if, for every interval [a, b] and every Caratheodory solution x{-) of 

x{t) e F{t,x{t)) te[a,b], (1.1) 

there exists also a solution of 

y{t)eextF{t,y{t)) te[a,b] (1.2) 

such that 

y{a) = x{a), y{b) = x{b). (1.3) 

If A, B are respectively n x n and n x m matrices, and U C M"^ is compact convex, the 
well known Bang-Bang Theorem [8, 10, 15] implies that the above property holds for the 
"linear" multifunction 

F(t, x) = {A{t)x + B{t)u; ueU] C iR". (1.4) 

In the present paper, the bang-bang property is proved for a new class of "concave" mul- 
tifunctions, characterized by the existence of suitable linear selections. The proofs rely 
on Lyapunov's theorem in connection with a Baire category argument. As applications, 
we obtain some closure theorems for the reachable set of a differential inclusion with non- 
convex right hand side, and new existence results for optimal control problems in Mayer 
as well as in Bolza form. 

Roughly speaking, the Baire category method consists in showing that the set 5^^*^ 
of solutions of (1.2) is the intersection of countably many relatively open and dense subsets 
of the family of all solutions of (1.1). Since is closed, Baire's theorem thus implies 
gextF _^ rpj^g effectiveness of such an argument, in connection with the Cauchy problem 
for a differential inclusion, was suggested by Cellina [5] and demonstrated in [4, 9, 19] and 
in other papers. Here, this basic technique will be combined with Lyapunov's theorem and 
applied to the two-point boundary value problem (1.2), (1.3). 
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The use of a Lyapunov-type theorem, in order to prove existence of optimal solutions 
for non-convex control problems, was introduced by Neustadt [12] and later applied in [1, 
13, 16] to a variety of optimization problems, always in connection with evolution equations 
and cost functionals which are linear w.r.t. the state variable. In [6], Cellina and Colombo 
showed that the linear cost functional can be replaced by one which is concave w.r.t. the 
state variable. Extensions and applications to partial differential equations have recently 
appeared in [7, 14]. We remark that, if a variational problem of the type considered in [6] is 
reformulated as a Mayer problem of optimal control, then the corresponding multifunction 
satisfies our concavity assumptions. The present results can thus be regarded as a natural 
extension of the theorem in [6], for optimization problems which are "fully concave": in 
their dynamics as well as in the cost functional. 



2 - Preliminaries. 

In this paper, | ■ | is the euclidean norm in IR^, B{x,r) denotes the open ball centered 
at X with radius r, while B{A, e) denotes the open e-neighborhood around the set A. We 
write A and WA respectively for the closure and the closed convex hull of A, while A\B 
indicates a set-theoretic difference. The Lebesgue measure of a set J C iR is meas{J). We 
recall that a subset A C S is a G5 if A is the intersection of countably many relatively 
open subsets of S. 

In the following, /Cn denotes the family of all nonempty compact convex subsets of 
iR", endowed with the Hausdorff metric. A key technical tool used in our proofs will be 
the function h : x /Cn 1— > -K U {—00}, defined by 

h{y, K) = sup I (^^' \f{x) - dxy ■ f: [0, 1] ^ K, f{x) dx = , (2.1) 

with the understanding that h{y,K) = —00 if y ^ K. Observe that h'^{y,K) can be 
interpreted as the maximum variance among all random variables supported inside K, 
whose mean value is y. From the above definition, it is clear that 

h{C + y, C + K)^ h{y, K), h{Xy, \K) = \h{y, K), G iR^ A > 0. (2.2) 



For the basic theory of multifunctions and differential inclusions we refer to [1] . Given 
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a solution x{-) of (1.1), following [3] we define its likelihood as 

L{x)^{^j h'^{x{t),F{t,x{t))d?j =\\h{x, F{-,x)\\c2. (2.3) 

The following results were proved in [3] : 

Lemma 1. For every y,K, one has h{y,K) < r{K), where r{K) is the radius of the 
smallest ball containing K (i.e., the Cehyshev radius). Moreover, h{y, K) — iffy G extK. 
Therefore, a solution x{-) of (1-1) satisfies also (1-2) iff L[x) — 0. 

Lemma 2. The map (y, K) i— > /j,(y, K) is upper semicontinuous in both variables and 
concave w.r.t. y. The map x{-) i— > L{x) is upper semicontinuous on the set of solutions of 
(1-1), endowed with the C° norm. 



3 - The main results. 

In the following, we denote by the set of all Caratheodory solutions of the two- 
point boundary value problem 

x{t) e F{t, x{t)), x{a) = p, x{b) = q. (3.1) 

Theorem 1. Let F : Mx JR" i— > 2"^" be a continuous multifunction with compact, convex 
values. The following conditions are equivalent: 

(1) For every interval [a, b] and every p,q& M^, if S^' j, ^ 0, then the set of solutions of 

y{t)eextF{t,y{t)), x{a) = p, x{b) = q (3.2) 

is a dense Gg in S^' j,. 

(2) F has the bang-bang property 

(3) For every interval [a, b] and every p,q E IR^, if S^'^ ^ 0, then for every £ > there 
exists a solution x{-) of (3.1) such that 

L^{x)= 1 h'^{x{t),F{t,x{t))) dt < s. (3.3) 

Proof. (1) (2) If ^ 0, then by (1) the set of solutions of (3.2), being dense, is 
nonempty. Hence (2) holds. 
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(2) (3) If Sl'l ^ then by (2) there exists a solution y(-) of (3.2). This imphes (3), 
because by Lemma 1 

b 

h''{y{t),F{t,y{t)))dt = 0<e. 

(3) =^ (1) Consider the sets = {x E S'^'J ; L{x) < By Lemma 2, L is upper 
semicontinuous, hence each Am is open. Now fix any x{-) e •S'^'^, £ > 0. Define 

n={{t,z); te[a,b], \z-x{t)\<e} 

and choose a constant M so large that 

F{t, x) c B{o, M) y{t, x) e (3.4) 

Split the interval [a,b] into k equal subintervals Jj = [tj-i, U], inserting the points ti = 
a+ {i/k){b — a), choosing k so large that 2M{b — a)/k < e. 

By the assumption (3), for each i there exists a solution j/j : [t^-i, ti] i— > of the 
two-point boundary value problem 

y{t) eF{t,y{t)), y{ti-i)^x{U-i), y{U) ^ x{U), (3.5) 

with ^ 

L2(y,) = / ^ /i'(yi(t), F(t,y,(t))) cZt < (3.6) 

Define y{-) as the solution of (3.1) whose restriction to each Jj coincides with j/j. Given 
any t e [a, 6], if, say, te Jj, then (3.4), (3.5) imply 

\y{t) - x{t)\ < ^ \yi{s) - x{s)\ ds < ^^^^"""^ < e. 

Hence |||/ — a;||co < £. Moreover, y e A^. because 

i=l '^ti-i 

Since x(-) and e > were arbitrary, this proves that each A^ is dense in S*^'^. By Baire's 
theorem, it follows that A = f]^ A^ is a Gs dense subset of S^'^. If j/ e A, then L(y) = 
and hence y{t) e extF{t,y{t)) almost everywhere. 

In the previous theorem, the implication (3) =^ (1) determines the strength of the 
category method. In order to prove that "most" solutions of (3.1) actually solve (3.2) as 
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well, it suffices to show (for every a,b,p,q) the existence of some solution of (3.1) with 
arbitrarily small likelihood. Roughly speaking, this requires the construction of some 
solution y of (3.1) whose derivative remains close to the extreme points of F{t,y) during 
most of the time. 

In practice, the condition (3) may often be easier to verify. We now show that this is 
indeed the case, if the multifunction F satisfies suitable concavity conditions. 

Theorem 2. Let F : M x i— > 2^" be a Hausdorff continuous multifunction with 
compact, convex values. Assume that: 

(Cl) For each {t,x) and every y e F{t,x), there exists a linear function z i— > Az + c 
satisfying 

y = Ax + c, Az + ceF{t,z) ^z eB{x, p{t,x)), (3.7) 
where the radius p = p{t, x) remains uniformly positive on compact sets. 

(C2) For each (t, x), every y G F{t, x) and e > 0, there exist S > and n+1 linear functions 
z I— > A'z + Ci, i = 0, . . . ,n, such that 

y ecd{A'x + co , ... , A'x + Cn}, (3.8) 

h{A'x + Ci, F{t,x)) <e Vz, (3.9) 

A'z + Ci e F{t,z) yz eB{x,S), Wi. (3.10) 

Then F has the bang-bang property. 

We refer to (Cl), (C2) as concavity conditions because they require, for each point 
(t, X, y) of the graph of F, the existence of suitable linear (non-homogeneous) selections. 
A similar property is shared by the epigraph of a concave scalar function, which admits 
global linear selections through each of its points. 

Proof of Theorem 2. We will prove that F has property (3) stated in Theorem 1. Let 
x*{-) be a solution of (3.1), for some interval [a, 6] and some points p,q & M^. Let any 
£ > be given, and define 

77= ini p{t,x%t)), V^{{t,zy, te[a,b], \z - x* {t)\ < rj} . (3.11) 

te[a,6] 
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By assumption, 77 > 0. Choose M so large that 



F{t, z) C S(0, M) V(t, z) e V. (3.12) 

By Lemma 1, this imphes 

h{y, F{t, z)) < M Vy, V(t, z) e V. (3.13) 

1. As a first step, we construct measurable, bounded functions A, c, such that 

x*{t) = A{t)x{t) + c{t) for a.e. t e [a,b], (3.14) 

A{t)z + c(t) e F(t, z) "iz e S(a;*(t), ry). (3.15) 

Since x*{-) is measurable, by Lusin's theorem there exists a sequence of disjoint compact 
sets {Jv>)v>i such that 

meas(\a,h] \ |J J^j = (3.16) 

and such that the restriction of x* to each J^, is continuous. Define the multifunction 
G : [a, 6] ^ 2^"'"x^" by setting 

G(t) = {(^,c): x*{t) = Ax + c, Az + ceF{t,z) ^z eB(x*(t),r])}. 

Because of (CI) and of the choice of rj, G{t) ^ for a.e. t. One easily checks that the 
restriction of G to each J^, has closed graph, because of the continuity of x* , x* and F. 
Hence, G is a measurable multifunction on [a, 6] with closed, nonempty values. By [11], 
it admits a measurable selection t ^— (A(t), c(t)), which clearly satisfies (3.14), (3.15). 
Observe that the matrices A{t) and the vectors c{t) must be uniformly bounded, because 
of (3.15), (3.12). 

2. As a second step, we construct measurable functions A' ,co, ■ ■ - CnjOo, ■ ■ ■ ,On,S, such 
that, for almost every t G [a,b], the following holds: 

n 

S{t)>0, ei{t)e[0,l], J]^i(0 = l, (3.17) 

n 

x*{t) = A'{t)x%t) + 5^^,(t)Q(t), (3.18) 
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A'{t)z + Ci{t) e F{t,z), h'^{A'{t)z + Ci{t), F{t,z)) <s Vz, ^z e B{x*{t),5{t)). 

(3.19) 

By Lemma 2, h is upper semicontinuous, hence there exists a nonincreasing sequence 
(^m)m>i of continuous functions such that 

h{y, K) = inf hm{y, K) V(y, K) e x /C„. (3.20) 

m>l 

For each m > 1, define the multifunction 

Hmit) = I {A',co,...,Cn,eo,---,On,S); S=—, ^ie[0,l], 

1^ m 

n n 

x''{t) = Y,UA'x{t)+ci), Y.^i = i: 

i=Q i=0 

A'z + ae F{t,z), h^{A'z + Ci, F{t,z)) <s e s(^x(t), ^) |. 

If {Ju)u>i is the same sequence of compact sets considered at (3.16), the continuity of 
X*, X*, hm and F imphes that the restriction of Hm to each J^, has closed graph, with 
uniformly bounded, possibly empty values. 

Defining 1^ = {t; H^it) ^ 0}, it is clear that on each the multifunction 
is measurable with closed, nonempty values. By [11], it admits a measurable selection, 
say t I— > ^m{t)- By (C2), (3.20) and the continuity of each hm, for every u we have 
Um>i-^^ — Therefore, the selection 

{A'{t), Co{t), Cn{t), 9o{t),..., 9r,{t), 6{t)) = $^(t) iff telm\[jl£ 

is measurable and defined for a.e. t e [a, 6]. By construction, the conditions (3.17)-(3.19) 
hold. 

3. We can now complete the proof of the theorem. Since 5{-) is measurable and positive, 
there exists an integer m* such that 

^<r/, meas{j'^*)>{h-a)-e, (3.21) 

TY}/ 

where 

J'm^{te[aM; m>-: \A'{t)\M{t)\<A- (3.22) 
I. m J 

Split [a, h] into k equal subintervals Ij = [tj-i, tj], inserting the points tj = a + {j /k){b — a) 
and choosing k so large that: 

2M^^ < — . (3.23) 
km* ^ ^ 



Using the selections A, c and A', Ci, 9i constructed in the previous steps, define: 

(A{t) if t^j:^,, 

A*it) = { 

[A'{t) if teJU*, 

(c{t) if t^JU*. 

m = { 

[^=omci{t) if teJ^.. 
By (3.14), (3.18), x*{t) = A*{t)x{t)+f{t). CaUing W{-, •) the matrix fundamental solution 
of the bounded linear system v = A*{t)v, we thus have the representation 

x*{t) = W{t,tj_i)x*{tj_^) + / W{t,s)f{s) ds, t e tj]. 

Applying Lyapunov's theorem on each interval Ij, for every j we obtain a measurable 
partition |/j o, • • ■ , -(j.n} of and an absolutely continuous function Wj satisfying the 
two-point boundary value problem 

( A* {t)wj (t) + c{t) if telj\ J^. , 

Wj{t) - <^ 

(A*{t)wj{t)+ce{t) if telj,£nj^,, £ = 0,...,n, 

{tj ) = X* {tj ) , Wj {tj+i) = X* {tj+l) . 

We claim that 

\wj{t) -x*{t)\<^ Vt e /-,-. (3.24) 

If not, there would exist a first time r e Ij such that 

K(t)-x*(t)| = ^. (3.25) 

Recalling (3.15), (3.19) and using (3.12) and (3.23), we then have: 

b-a 1 

\Wj{t) -x^{t)\ dt < 2M 

'tj-i 

a contradiction with (3.25). This proves (3.24). In particular, by (3.15), (3.19) we conclude 
that Wj{t) e F(t, Wj{t)) for a.e. t e Ij. 

Now consider the solution w{-) of (3.1) whose restriction to each Ij coincides with Wj. 
Recalling (3.13), (3.19), (3.21), its likelihood is computed by 



\wj{T) -x*{t)\< / \wj{t) - x*{t)\ dt < 2M^^ < 



L^{w)= I h'^{w{t),F{t,w{t))) dt+ [ h'^{w{t),F{t,w{t))) dt 

< e ■ meas{J'^.) + • meas{[a, h] \ J^*) < e{{h - a) + M^). 



Since s was arbitrary, this establishes the property (3) in Theorem 1, which is equivalent 
to the bang-bang property. 

Remark 1. The previous theorems, with the same proofs, remain valid if F is defined on 
some open set C iR x IW^. 



4 - Examples of concave mult ifunct ions. 

Aim of this section is to exhibit some classes of multifunctions which satisfy the 
concavity properties (CI), (C2) stated in Theorem 2. 

Proposition 1. Let Lp : M x i— >]0, oo[ be a continuous function, with x i— >• (p{t,x) 
convex for every t. Let U C IRJ^ be compact, convex, containing the origin. Then the 
multifunction 

F{t,x) ^<p{t,x)U (4.1) 

has the bang-bang property. 

Proof. In order to apply Theorem 2, we first verify the concavity condition (CI). Fix any 
(tjx) and any y = {p{t,x)u G F{t,x). Since Lp is continuous and strictly positive and its 
subdifferential d^^ w.r.t. x is uniformly bounded on compact sets, we have 

^{t,x)+i-{z-x)>Q eda:ip{t,x), Wz eB{x, p{t, x)), (4.2) 

for some function p = p{t, x) uniformly positive on compact sets. Choose any vector 
^ e dx(fi{t, x) and define the linear map 

z I— > Az + c = (^ • z)u + [(fi{t, x)u — (^ • x)u\ . 

li \z — x\ < p{t, x), we need to show the existence of some uj e U such that 

(^ • z)u + [(f{t, x)u - (^ • x)u] = (f{t, z)uj 

From (4.2) and the convexity of (p it follows 

(f(t, x) + ^ ■ (z - x) 
(f{t, z) 

for some a e [0, 1]. The assumptions on U thus imply oj e U. 
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(4.3) 
(4.4) 



We now turn to the condition (C2). Let (t, x), y = (p{t, x)u e F{t, x) and £ > be given. 
We can assume 

1/ V 

Y^^iUi, ^ie(0,l], Y.^i = l (4.5) 



V 

U 



for some e {1, . . . , n + 1}, ttj e extU . Select ^ e dx(p{t, x) as before and define 

u'^ = Ui+e'{u-Ui), (4.6) 

choosing £' e (0, 1) so small that 

This is possible because h{ui, U) — and h is upper semicontinuous. Then define 

A'z = (^ • z)u, Ci = (p{t, x)u[ - (C • x)u. 
Recalling (2.2)2, (4.7) yields 

h[A'x + Ci, F{t,x)) = h[(f{t,x)u[, (p{t,x)U) < e. 
Hence, for 2; in a small neighborhood of x, the upper semicontinuity of h implies 

h{A'z + Ci, ifi{t,z)U) < e Vz. 

Moreover, by (4.5), (4.6), 

y = (fit, x)u = Oi(fi{t, x)ui e co\^A'x + Ci, i = 1, . . . ,1/^. 

i=l 

It remains to prove that A'z + Cj e (p{t, z)U for \z — x\ small enough. For each fixed i, 
define the subspace Ei = span{u, Ui}. 

If Ei has dimension 2, consider the triangle A = co{0, u, Ui} and call rii, n2, the 
unit vectors in Ei which are outer normals to the sides Ui — u, Ui, u, respectively. Observe 
that 

(p{t, z)cd{0, u, Ui] ^ {y e Ei- ni-y < (p{t, z){yvi-u), n2 ■ 2/ < 0, ns ■ y < O} C F{t, z). 

(4.8) 

Since u'^ is a strict convex combination of u and Ui, one has 112 ■ u'^ < 0, xi^ ■ u'^ < 0. By 
continuity, for z sufficiently close to x we still have 

iij • [ip{t, x)u'i + C • (-Z - x)u] = Ta.j ■ [A'z + c^] < 0, j = 2, 3. (4.9) 
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Moreover, since ni ■ u = ni • u[ > 0, the convexity of cp implies 

(p{t, z){ni ■ u) > [(p{t, x) + ^- {z- x)] (ni ■ u) 

= m • [<fi{t, xX + (e • (-2 - (4.10) 
= ni • [A'z + Ci]. 

By (4.8), the inequahties (4.9), (4.10) together imply A'z + Ci e F{t, z). 

Finally, consider the case where Ei has dimension < 1. Then, either u = Ui, hence 
1^ = 1 and u e extF{t,x). In this case, the same argument as in (4.3), (4.4) can be used. 
Or else u'^ lies in the relative interior of the segment S = co{0, u, Ui\. In this case, the map 
z I— > A' z + Ci takes values inside E'j, with </'~^(t, x) \A'x + G rel intS. By continuity, 
(p~^ {t, z) [A'z + Cj] E S for 1 2; — x\ small enough. This proves again that A'z + Q e F{t, z) 
for every 2; in a neighborhood of x. 

An application of Theorem 2 now yields the desired conclusion. 

The next application is concerned with a multifunction F whose values arc polytopes, 
with variable shape but constant number of vertices. More precisely, we assume that 
F{t, x) admits the representations 

F{t,x) =cd{yi{t,x),...,yN{t:X)}, (4.11) 

where yi,. . .,yN are the (distinct) vertices of F{t, x), as well as 

F{t,x) = \y e M"^ ; Wj{t) ■ y < 'ipj{t, x) = max Wj{t)-u, j = l,..,k}. (4.12). 

On the product set of indices {1, . . . , N} x {1, . . . , /c}, we consider the "incidence" relation 

ir^j iff Wj{t) -yiit.x) ^ i;j{t,x). (4.13) 

Proposition 2. Let F be a multifunction admitting the representations (4-11), (4:12). 
Assume that 

(i) Wj : 1R \-^ IR^ , i/jj : M X IR^ 1— > M are continuous functions, \wj{t)\ = 1, each map 
X I— >• '0j(^: is convex. 

(a) The relation ~ defined at (4-13) is independent of {t,x). 
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Then F has the bang-bang property. 



Proof. For each i e {1, . . . , N}, consider the set of indices 

Ji = {j; Wj{t) ■yi{t,x) =ijjj{t,x)}. 



(4.14) 



By (ii), this set does not depend on t, x. 

We begin by checking the condition (CI) in Theorem 2. First, assume y e extF{t, x), 
say y = yi{t, x). In this case we can choose n independent vectors, say Wj^ (t), . . . , Wj^ (t), 
with ji, ■ ■ ■ ,jn G Ji- Define the dual vectors w*^, requiring that 



if £ = m, 
if e^m. 



(4.15) 



By convexity, each function z ^ (t, z) is diff'erentiable almost everywhere. Therefore, 
there exists a sequence of points x^^ ^ x such that the gradients Vxi^{ti x^,) exist for each 
V, together with the limits 



lim VxiJj{t,Xjy) = e dxipj{t,x) 



Now define 



)w 



3i' 



= Y^i^h ■ z)w]v = yi{t, x) - Y^iiu ■ 

1=1 i=i 
Clearly, AiX + Ci = yi- Using the representation (4.12) we now check that 

AiZ + Ci e F{t,z) \/z eB{x,p{t,x)) 



(4.16) 



(4.17) 



(4.18) 



for some p = p{t, x) uniformly positive on compact sets. 

If j e Ji, then there exist unique coefficients such that Wj{t) = '^a£Wj^{t). The 
assumption (ii) together with (4.16) now implies 

n 

ipj {t, z) = Wj (t) ■ yi {t, z) = Y^ a^ipj^ {t, z) Mz, 



^aiij^ e dx^jit^x). 



(4.19) 



1=1 



Prom (4.19) and the convexity of V'j it follows 

n 

Wj (t) ■ [AiZ + Ci\ = ^ aiWj^ {t) 



e=i 



h=l 



n 

= ^pj{t, x)+ (^Yl ai^j^^ -{z-x) < ijj{t, z). 
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On the other hand, if j ^ Jj then Wj{t) • [AiX + Cj] < tl>j{t,x). Hence, by continuity we 
still have 

Wj{t) • [AiZ + Ci] < 'ipj{t, z) \/z e B{x,p{t, x)). 

By the assumption (ii), the continuity of the functions Wj, ijjj and the local boundedness of 
the subgradients d^^ipj, it follows that p can be taken uniformly positive on bounded sets. 
This proves (CI) in the case y G extF{t,x). 

When y is an arbitrary element in F(t, x), there exist extreme points yi and coefficients 
9i e [0, 1] such that 

N N 

y = ^Oiyi{t,x), ^6*^ = 1. 

If Ai^Ci are the matrices and vectors defined at (4.17), then the convex combinations 
A=Y, ^i^i, c=Y^ 9iCi satisfy 

Ax + c = y, Az + ce F{t,z) \/z eB{x, p{t,x)). 



Next, consider the condition (C2). Let {t,x), y e F{t,x), £ > be given. Write y as 
a convex combination of points yi, . . . ,yi, e extF{t, x), say 

y^Y^^rVi ^^e(o,l] Y.^i^\ 

and define 

y'i ^yi + £'{y -yi), 

choosing e' e (0, 1] so small that 

h{y'„ F{t,x))<e Vi. (4.20) 

Consider the vector space 

E = spanjwj (t) ; wj (t) ■ y = il^j {t, x) j . 

Choose a basis {wji, . . . ,Wj^} of E and define the dual basis {wj^, . . . , w*^} as in (4.15). 
Select vectors e dxipjit,x) as in (4.16) and define 

A'z = J^i^j, ■ z)wl, Ci=yi- J](0, • x)wl. (4.21) 

e=i e=i 
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The above definitions imply 

V 

y = ^y'i ^co{A! X ^ Ci\ z = i,...,//}. 

Moreover, by (4.20) and the upper semicontinuity of /i, for 1^ — a;| small enough we have 

h{A!z^Ci, F{t,z)) < e 
Using the representation (4.12), we now prove that A'z + Cj e F(t, z). If Wj{t) e then 



Wj {t)-y = Wj (t) ■ yl = ipj {t, x) 



Vz = 1, . . . 



Moreover, there exist coefficients such that Wj{t) — ^ct^Wj^(0- The assumption (ii) 
together with (4.16) now implies 



ipj {t, z) = Wj it) ■ Vi {t, z) = ^ aii^j^ (t , z) yz, 



(4.22) 



(4.23) 



£=1 



From (4.22), (4.23) and the convexity of ipj it follows 



h=l 



Uj{t) ■ [A'z + Ci] = ^ aiWj^{t) ■ 
e=i 

< i^j{t,z). 
On the other hand, if Wj{t) ^ E, then 

Wj{t) ■ [A'x + Ci] = Wj{t) ■ y[ < ipj{t, x). 

By continuity, for 1^; — x| sufficiently small we still have 



Wj{t) ■ [A'z + Ci] < ipj{t,z). 

This completes the proof of condition (C2). An application of Theorem 2 now yields the 
desired result. 
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Remark 2. Assume that b are a n x n matrix and a n-vector, depending continuously on 
t, and that F is a continuous, compact convex valued multifunction satisfying the concavity 
conditions (CI), (C2). Then the multifunction 

G{t, x) = A{t)x + h{t) + F{t, x) 

satisfies all assumptions in Theorem 2 as well. In particular, from Proposition 1 it follows 
that the bang-bang property holds for a control system of the form 

x = A{t)x + h{t) + (f{t,x)u, u{t)eU, 

with U compact, convex, containing the origin and (f > convex w.r.t. x. 



5 - A nonconvex optimal control problem. 

This section is concerned with an application of Theorem 2 to an optimal control 
problem of Bolza. The analysis will clarify the connections between the concavity condi- 
tions (CI), (C2) and the assumptions made in [6, 12]. Given the linear control system on 

x{t) = A{t)x + f{t,u{t)) u{t)eU (5.1) 
with initial and terminal constraints 

x{0) = x {T,x{T))eS, (5.2) 

consider the minimization problem: 

min [ a{t,x{t)) + (3{t,u{t)) dt. (5.3) 
Jo 

Theorem 3. Let the functions A,f,a,f3 be continuous, with a concave w.r.t. x. Assume 
that the control set U C IR^ is compact and that the terminal set S is closed and contained 
in [0, To] X IR^ , for some Tq. If some solution of (5.1), (5.2) exists, then the minimization 
problem (5.3) admits an optimal solution. 

Proof. We begin by adding an extra variable xq., writing the problem in Mayer form: 

mina;o(T) (5.4) 
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x{t) = A{t)x{t) + f{t,u{t)) 

u(t) e U, (5.5) 

xo{t) = a{t,x{t)) + I3{t,u(t)) 

{x, xo){0) = {x, 0) (T, x{T)) e S. (5.6) 

The continuity of A, f and the compactness of U imply that all trajectories of the differ- 
ential inclusion 

x{t) e A{t)x{t) +co{f{t,u); ueU}, x{0) ^ x, te[0,To], (5.7) 
are contained in some bounded open set O C iR x IR^. Define the constant 

M = l + sup{a{t,x)+P{t,u); {t,x)en, u e U} (5.8) 
and the multifunction (independent of xq) 

F{t,x,XQ) = cd{{y,yo); y = A{t)x+f{t,u), a{t, x)+l3{t,u) <yo < M for some u eU}. 

(5.9) 

Observe that F admits the representation 

{n+l n+1 
(y, yo); y = A(t)x + Vo = [a{t, x) + J2 ^iPi^, «i)] (1 - ^;) + Mv, 

{do,...,en+i) e An+i, uo,...,Un+ieU, ve[0, 

(5.10) 

where 

n+l 

An+1 = {(^0, . . . , On+i); e,e[0,l], ^ = l}. 

1=0 

Since F is continuous with compact convex values, the optimization problem (5.4) 
subject to the boundary conditions (5.6) with dynamics 

{x,xo)eF{t,x) (5.11) 

admits an optimal solution. The existence of a solution to the original problem (5.1)- 
(5.3) will be proved by showing that the multifunction F has the bang-bang property, for 
{t,x,XQ) eQ X M. 

To verify the concavity condition (CI), define the constant p > by 

- = sup{|e|; ^edMt,x), {t,x)eQ}. (5.12) 
P 

17 



Given {t,x) e ft, {y,yo) e F{t,x), assume first yo < M — 1. Choose any ^ e dxa{t,x) and 
consider the hnear map 

^z) = {y + A{t){z -x), yo + ^-{z-x)). (5.13) 
Prom (5.12) and the assumption on yo it follows that 

yo + ^-{z-x)<M yzeB{x,p). (5.14) 
Let y, yo be as in (5.10), for some 9i, Ui, v. Using the concavity of a we then obtain 

n+l 

y + A{t){z -x) = A{t)z + ^ifit^ 

n+l 

yo + ^-{z-x)> a{t, x) + Y^ ^^f^i^, Ui) + ^ ■ {z - x) 

i=0 
n+l 

>a(t, z) + J2Wt, Ui)- 

This, together with (5.14), implies ^{z) e F{t,z). 

Next, assume yo e [M — 1, M]. Then the definition (5.8) implies that the map 

^{z) = (y + A{t){z-x), yo) 

is again a selection of F(t, ■). 

We now turn to the condition (C2). Fix {t,x) E fl, y = (y, j/o) £ F{t,x), £ > 0, and 
choose ^ e dxa{t, x). Write y as a convex combination 

with xjj e extF{t,x), and define 

y'j = yj + £'{y -Vj), 

choosing e' e (0, 1] such that 

h{y'j, F{t,x))<s Mj. (5.15) 

We now distinguish two cases. 
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If Ho < M, then y'^ = {y'jjy'oj) satisfies yQj < M for all j. Hence, for \z — x\ sufficiently 
small, the maps 

^ji^) = {y'j + Mt)iz - x), y'oj + ^-{z-x)) 
are affine selections of F. Moreover, (5.15) and the upper semicontinuity of h imply 

h{^z,zo), F{t,z)) <e 

for all 2; in a neighborhood of x. 

On the other hand, if yo = M, then yoj = j/g ^ = M for all j. Hence the maps 

^j{z)^{y'^+A{t){z-x), M) 

are affine selections of F and satisfy (5.16) for \z — x\ sufficiently small. 

Applying Theorem 2, we now obtain the existence of an optimal solution {x*^Xq) : 
[0,T] ^ mj" xMto (5.4), (5.6), (5.11), with the additional property 

{x*, xl){t) e extF{t,x''{t)) for a.e. t G [0,T]. 

The representation (5.10) and the selection theorem [11] now yield the existence of some 
measurable tt* : [0, T] 1— > t/ such that 

x*it)=A{t)x*{t)+fit,u*it)), 
x*o{t) e {a{t,x*{t)) + /3{t,u*{t)), M] 

for almost every t. Since the terminal value Xq{T) is minimized, by (5.8) we must have 
XQ{t) < M almost everywhere. Therefore, x* is an optimal trajectory for the original 
system (5.1), corresponding to the control u*. 



6 - A counterexample. 

The following example shows how the bang-bang property may fail, if some of the 
assumptions in Theorem 2 or in Proposition 2 are not satisfied. More general results 
concerning systems of this form can be found in [17, 18]. 

On M^, consider the control system 

x{t) = f{x{t))+g{x{t))u{t), u{t) e [-1, 1], (6.1) 
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with 

f{xi,X2) = {xi,X2), g{xi,X2) = {l,Xi). 

For t G [0, 1], the trajectory t i-^ (0, e*), corresponding to the control u{t) = 0, steers the 
system from p = (0, 1) to q = (0, e). 
Defining the aiixiliary function 

2 

X 

V{xi,X2) = a;2 - y, 
a straightforward computation yields 

fv{x{t)) = V{x{t)) - ^ 
for every solution of (6.1). This implies 

V{x{t)) < e^V{x{0)), 

with equality holding if and only if xi{s) = for all s e [0,t\. In particular, the control 
■u = is the only one which steers the system from p to in minimum time. Hence the 
multifunction 

F{xi,X2) = {{xi+u, X2 + xiu); |tt| < l} 
does not have the bang-bang property. Observe that 

(i) For each y = f{x) + g{x)uj e F{x), defining 

one checks that the condition (CI) in Theorem 2 holds. However, the condition (C2) 
here fails. 

(ii) The multifunction extF{x) = {f{x) +g{x), f{x) — g{x)^ satisfies both (CI) and (C2) 
in Theorem 2, but its values are not convex. 

(iii) Each set F{x) is a segment. Moreover, F admits the representation 

F{x) = {y; w-y < ipyjix) = max w ■ (f{x) + g{x)u) }. 

|u|<l 

Since /, g are linear, each is convex. However, this representation does not satisfy 
all assumptions in Proposition 2. 
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